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Abstract 

Upstream reciprocity (also called generalized reciprocity) is a putative mechanism for cooperation in 
social dilemma situations with which players help others when they are helped by somebody else. It 
is a type of indirect reciprocity. Although upstream reciprocity is often observed in experiments, most 
theories suggest that it is operative only when players form short cycles such as triangles, implying a small 
population size, or when it is combined with other mechanisms that promote cooperation on their own. 
An expectation is that real social networks, which are known to be full of triangles and other short cycles, 
may accommodate upstream reciprocity In this study, I extend the upstream reciprocity game proposed 
for a directed cycle by Boyd and Richerson to the case of general networks. The model is not evolutionary 
and concerns the conditions under which the unanimity of cooperative players is a Nash equilibrium. I 
show that an abundance of triangles or other short cycles in a network does little to promote upstream 
reciprocity. Cooperation is less likely for a larger population size even if triangles are abundant in the 
network. In addition, in contrast to the results for evolutionary social dilemma games on networks, 
scale-free networks lead to less cooperation than networks with a homogeneous degree distribution. 

Introduction 

Several mechanisms govern cooperation among individuals in social dilemma situations such as the pris- 
oner's dilemma game. Upstream reciprocity, also called generalized reciprocity, is one such mechanism 
in which players help others when they themselves are helped by other players. It is a form of indirect 
reciprocity, in which individuals are helped by unknown others and vice versa [l][2]- 
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Cooperation based on upstream reciprocity has been observed in various laboratory experiments. 
Examples include human subjects in variants of the trust game, which is a social dilemma game [3HS], 
human subjects participating in filling out tedious surveys [S], and rats pulling a lever to deliver food to 
a conspecific [7]- Even more experimental evidence is available in the field of sociology in the context of 
social exchange l8l[9] (also see [TOl[Tl] for classical examples of the Kula ring). 

Nevertheless, theory and numerical simulations have revealed that upstream reciprocity in isolation 
does not promote cooperation (but see Barta et al. |12| for an exception). Upstream reciprocity usually 
supports cooperation only when combined with another mechanism that can yield cooperation on its own. 
Cooperation appears when the population size is small |131I14| . upstream reciprocity is combined with 
direct reciprocity or spatial reciprocity |15| , players move across groups |16j , players interact assortatively 
[17] . or players inhabit heterogeneous networks jl8) . 

In their seminal study, Boyd and Richcrson analyzed an upstream reciprocity game on a directed 
cycle and showed that it yields cooperation only when the cycle is small [13] . The shortest possible cycle 
with indirect reciprocity consists of three players (Fig. [1} because a cycle composed of two players only 
involves direct reciprocity. Cooperation is intuitively less likely for longer cycles because a player that 
helps a unique downstream neighbor on the cycle has to "trust" too many intermediary players for their 
tendency to cooperate before the player eventually receives help. 

Real social networks are full of short cycles represented by triangles, a feature known as transitivity |19| 
or clustering [^DH^ . Therefore, a natural expectation is that larger networks with a high level of 
clustering (i.e., many triangles) may facilitate cooperation based on upstream reciprocity [5]. In the 
present study, I address this issue theoretically. I extend the model of Boyd and Richerson [T3] to general 
networks and derive the condition under which the unanimity of players using upstream reciprocity is 
resistant to invasion of defectors. Then, I apply the condition to model networks to show that clustering 
does little to promote cooperation except in an unrealistic network. This conclusion holds true for both 
homogeneous and heterogeneous networks, where heterogeneity concerns that in the degree, i.e., the 
number of neighbors for a player. 

My results seem to contradict previous results for spatial reciprocity in which clustering enhances 
cooperation in the prisoner's dilemma game |23| and those for heterogeneous networks in which hetero- 
geneity enhances cooperation in various two-person social dilemma games [24H27) and in the upstream 
reciprocity game [18] . These previous models are evolutionary, however, whereas mine and the original 
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model by Boyd and Richcrson |13| are nonevolutionary and based on the Nash equihbrium. I opted to 
use a nonevohitionary setting in this study because interpretation of evolutionary games seems elusive 
for heterogeneous networks [551[5S] (see Discussion for a more detailed explanation). 

Results 

Preliminary: upstream reciprocity on a directed cycle 

Boyd and Richcrson proposed a model of upstream reciprocity on the directed cycle |13] . By analyzing 
the stability of a unanimous population of cooperative players, they showed that cooperation is unlikely 
unless the number of players, denoted by N, is small. 

In their model, the players are involved in a type of donation game. Each player may donate to a 
unique downstream neighbor on a directed cycle at time t — by paying cost c(> 0). The recipient of 
the donation gains benefit b{> c). Among the recipients of the donation at i = 0, those who comply 
with upstream reciprocity donate to a unique downstream neighbor at t = 1 by paying cost c. Chains of 
donation arc then carried over to downstream players, who may donate to their downstream neighbors at 
t = 2. At t = 1, defectors that have received a donation at t = terminate the chain of donation. Such 
defectors receive benefit 6 at t = and lose nothing at t = 1. This procedure is repeated for all players 
until all the chains of donation terminate. If all the players perfectly comply with upstream reciprocity, 
the chains never terminate. In contrast, if there is at least one defector, all the chains terminate in finite 
time. 

As in iterated games [501151] . w {0 < w < 1) is the probability that the next time step occurs. We can 
also interpret w as the probability that players complying with upstream reciprocity do donate to their 
downstream neighbors, such that they erroneously defect with probability 1 — in each time step. Each 
player's payoff is defined as the discounted sum of the payoff over the time horizon. In other words, the 
payoff obtained at time t (> 0) contributes to the summed payoff with weight w*. 

It may be advantageous for a player not to donate to the downstream neighbor to gain benefit b 
without paying cost c over the time course. However, a player that complies with upstream reciprocity 
may enjoy a large summed payoff if chains of donation persist in the network for a long time. 

Each player is assumed to be of either classical defector (CD; termed unconditional defection in [T5] ) 
or generous cooperator (GC; termed upstream tit-for-tat in [S]). By definition, a CD does not donate to 
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the downstream neighbor at t = and refuses to relay the chain of donation received from the upstream 
neighbor to the downstream neighbor at i > 1. A GC donates at t = and donates to the downstream 
neighbor if the GC received a donation from the upstream neighbor in the previous time step. 

For this model, Boyd and Richerson obtained the condition under which the unanimity of GCs is 
robust against the invasion of a CD (i.e., conversion of one GC into CD). When all players are GC, the 
summed payoff to one GC is equal to 

{b-c){l + w + w^' + ...) = ^—^. (1) 

If iV — 1 players are GC and one player is CD, the unique CD's summed payoff is given by 

bil + w + w' + ... + w^-')=^-^^^-^^^. (2) 

1 — w 

Therefore, GC is stable against the invasion of CD if the right-hand side of Eq. ([T]) is larger than that of 
Eq. (O, that is, 

w^-' > \. (3) 

Equation generalizes the result for direct reciprocity [nOlISI], which corresponds to the case where 
N = 2. Equation ^ also implies that cooperation is likely if u" is large. However, maintaining cooperation 
is increasingly difficult as N increases. 

Model 

I generalize the Boyd-Richerson model on a directed cycle to the case of general networks. Consider a 
network of N players in which links may be directed or weighted. I denote the weight of the link from 
player i to j by Aij > 0. I assume that the network is strongly connected, i.e., any player is reacheable 
from any other player along directed links. Otherwise, chains of donation starting from some playes never 
return to them because of the purely structural reason. In such a network, it would be more difficult to 
maintain cooperation than in strongly connected networks. Even for strongly connected networks that 
might accommodate upstream reciprocity, I will show that cooperation is not likely for realistic network 
structure. 

Assume that all the players are GC and that each GC starts a chain of donation of unit size at t = —oo. 
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Therefore, the total amount of donation flowing in the network in each time step is equal to N. In the 
steady state, the total amount of donation that each player receives from upstream neighbors is equal to 
that each player gives to downstream neighbors in each time step. I denote the total amount of donation 
that reaches and leaves player i by Nvi, where X^iLi ~ 1- this situation, the amount of donation 
that player i imparts to player j in each time step is equal to NviAij /k°^'^ , where = X^fci is the 
outdegree of player i. Player i receives payoff {h — c)Nvi in each time step. 

In our previous work |18| . we assumed that each GC starts a unit flow of donation at t = 0. In the 
present study, however, I wait until the flow reaches the steady state before starting the game at t = 0. 

The definition of CD for general networks is straightforward; a CD donates to nobody for t > 0. I 
extend the concept of GC to the case of general networks as follows. On a directed cycle, a GC quits 
helping its downstream neighbor once the GC is not helped by the upstream neighbor [13] . On a general 
network, the total amount of donation that GC i receives per unit time in the absence of a CD is equal 
to Nvi = X^fci ^"f-'eAu/kf^*". If there is a CD, the total amount of donation that GC i receives may 
be smaller than the amount that player i would receive in the absence of a CD. By definition, the GC 
responds to this situation by relaying the total amount of the incoming donation proportionally to all its 
downstream neighbors in accordance with the weights of the links outgoing from player i. 

As an example, suppose that one upstream neighbor of GC i, denoted by j, is CD and all the other 
iV — 1 players, including player z, are GC. At t = 0, the total amount of donation that i receives is 
equal to "^^^^^ NviAu/kf^^ ^ which is smaller than Nvi. Player i donates Nvi in total. Therefore, 
player Vs payoff at t = is equal to b^^^^ NvtAu/k'p*' ~ cNvi. In response to the amount of 
donation that player i received at t = 0, player i adjusts the total amount of donation that it gives 
the downstream neighbors from Nvi to "^^^i i-^j Nvf Au/k'^^*' at t = 1. Therefore, player i donates 
"^f^i £-ij{NviAii/k^^^) X {Aiji/k°^^) to its downstream neighbor j'. This quantity is smaller than the 
donation that player i would give player j' in the absence of CD j, which would be equal to NviAij/ /fc,°"'. 

An implicit assumption is that the GC cannot identify the incoming links along which less donation 
is received as compared to the case without a CD. In other words, even if a GC is defected by the CD in 
the upstream, the GC cannot directly retaliate. Instead, the GC distributes the retaliation equally (i.e., 
proportionally to the weight of the link) to its downstream neighbors. 
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Stability of upstream reciprocity in networks 

In this section, I derive the condition under which no player is motivated to convert from GC to CD 
when aU the players are initially GC. 

The steady state v = {vi ... v^f) is equivalent to the stationary density of the simple random walk 
in discrete time. It is given as the solution of 

V = vD-^A, (4) 

where A = {Aij) is the TV- by- TV adjacency matrix, where Aij represents the weight of the link from i to 
j, and the diagonal matrix D is defined as D ^ diag(fcj^*, . . . , fc^*). The (i, j) element of D^^A is equal 
to Aij/k°^^, that is, the probability that a walker at node i transits to node j in one time step. If the 
network is undirected, the solution of Eq. Q is given by Vi = ki/J2eLi ^e, where ki = = J2eLi — 

The summed payoff to player i is equal to 

f:{b-c)w*Nu^=^J^-^. (5) 
^-^ 1 — w 

t=o 

To examine the Nash stability of the unanimity of GC, I analyze the situation in which player i is 
CD and the other TV — 1 players are GC. At i — 0, the — 1 GCs pay cNvj {j ^ i), and player i pays 
nothing. Therefore, the benefits to the N players, including player i, at t = arc given in vector form by 

bNv{I - E,)D-^A, (6) 

where / is the N-hy-N identity matrix, and Ei is the A^-by-A^ matrix whose (i, i) clement is equal to one 
and all the other elements are equal to zero. The benefit to player j (l < j < N) at t = is equal to the 
jth element of the row vector given by Eq. 

At t = 1, the downstream neighbors of player i donate less because player i defects at t = 0. The 
amount of donation given to player j, where j is not necessarily a neighbor of i, at t = is equal to the 
jth element of the row vector Nv{I — Ei)D^^A. Therefore, the total amount that GC i) donates 
to its downstream neighbors at t = 1 is equal to the jth element of Nv{I — Ei)D^^A. Player i, who is 
CD, does not donate to others at t = 1. Therefore, the amount of the donation issued by the players at 
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t = 1 is represented in vector form as Nv{I — Ei)D ^A{I — Ei). The discounted benefits that the players 
receive at t = 1 are given in vector form by 



bNv [{I ~ E,)D-^Ay 



(7) 



By repeating the same procedure, we can obtain the summed benefits to the players in vector form as 



bNv [(-^ - Ei)D-^A] = bNv{I - Ei)D-^A [l - w[I - E,)D-^A] ^ 



(8) 



To derive Eq. (jS)), I used the fact that the spectral radius of w{I — Ei)D^^A is smaller than unity (that 
of D^^A is equal to unity). The ith element of Eq. ([5]) is equal to the summed payoff to player i because 
player i does not pay the cost to donate at any t. 

If the ith element of Eq. ([5]) is smaller than the quantity given by Eq. ([S]), player i is not motivated 
to turn from GC to CD. Therefore, the unanimity of GC is stable if and only if 



bNv{I - E^)D-^A [I - w{I - E^)D~^A] ' 



(b - c)Nv, , 
\ — w 



(9) 



where \i indicates the ith element of a vector. By rearranging terms of Eq. ([9|), I obtain 



«[/-(/- E,)D-^A\ ■ [I - w{I - E,)D-^A] ' 



> -v^ {l<i< N). 
b 



(10) 



Because vD ^A = v, Eq. PH)) can be reduced to 



Ail 

ki 



A, 



N 



[l-wiI-E,)D-'A] ' 



> - {l<i<N). 



(11) 



Equation ([Tl]) is never satisfied when w ^ because An = 0. It is always satisfied when u; w 1 
because the left-hand side of Eq. (jlOp tends to Vi as w — 1. 



For a directed cycle having N nodes, v ^ {1 ... 1)/N, k°^^ = 1 (1 < i < N), and Aij is equal to 1 if 



(i + 1) mod N — j and otherwise. Owing to the symmetry with respect to i, we only have to consider 
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the condition (i.e., Eq. © or Eq. ((TT|) ) for player 1 and obtain the foUowing: 



V 



[{I-E,)D-'A] 



V 



[{I-E,)D-^A]^ 



v{I - E,)D 




(13) 



(14) 



(12) 



[{I-E,)D~'A\ 



N 



(0 ... 0). 



(15) 



V 



Therefore, Eq. pT|) can be read as ^ > c/b, which reproduces the result by Boyd and Richerson |13) . 
Numerical results for various networks 



cally difficult because this matrix may have nondiagonal Jordan blocks. Standard formulae for decompos- 
ing matrices under independence of different eigenmodes do not simply apply. The method for efficiently 



I conducted numerical simulations for different networks to determine the threshold value oiw, denoted 
by Wth, above which the unanimity of GC is stable against invasion of CD. The conclusions derived from 
the following numerical simulations are summarized as follows: (a) abundance of triangles (and other 
short cycles) hardly promotes cooperation, and (b) networks with heterogeneous degree distributions 
yield less cooperation than those with homogeneous degree distributions. 

Network models 

I use five types of undirected networks generated from four network models. It would be even more 
difficult to obtain cooperation in directed networks because undirected networks generally allow more 
direct reciprocity than directed networks (see Discussion for a more detailed explanation). 

The regular random graph (RRG) is defined as a completely randomly wired network under the 
restriction that all nodes (i.e., players) have the same degree k [2Tll22| . The RRG has low clustering (i.e., 
low triangle density) and is homogeneous in degree [2Tl[22ll32] . 

To construct a network from the Watts-Strogatz (WS) model [321, nodes are placed in a circle and 
connected such that each one is adjacent to the fc/2 closest nodes on each side on the circle. In this way. 



For general networks, calculating [/ — w{I — Ei)D , which is used in Eqs. ^ and pT|) . is techni- 



calculating [/ - w{I - E,)D-^A] 



-1 



is described in the Methods section. 
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each node has degree k. A fraction p of the hnks is then rewired, and a selected hnk preserves one of its 
end nodes and abandons the other end node. Then, I randomly select a node from the network as the 
new destination of the rewired link such that self-loops and multiple links arc avoided. I use two cases, 
one in which p = and the other in which p is small but greater than zero. In both cases, the network 
has a high amount of clustering. When p = 0, the network is homogeneous in degree and unrealistic 
because it has a large average distance between nodes. When p is positive and appropriately small, the 
degree is narrowly distributed and the network has a small average distance [32j . 

As an example of networks with heterogeneous degree distribution, I use the Barabasi- Albert (BA) 
model. It has a power-law (scale- free) degree distribution p{k) oc k^^, a small average distance, and low 
level of clustering [20l[33] . 

To probe the effect of triangles in scale-free networks, I use a variant of the Klemm-Eguiluz (KE) 
model [341135] . For appropriate parameter values, my variant of the KE model generates scale- free 
networks with p{k) oc k^^, small average distances, and a high level of clustering. 

The effect of clustering 

For a fixed network and a fixed value of cost-to-benefit ratio c/6, the threshold value of w above which 
the unanimity of GC is stable against conversion of player i into CD depends on i. I denote this value 
by Wthii)- I determine wth as the largest value of u)th(*) (1 < « < N). This is true because once a certain 
player i turns from GC to CD, other players may be also inclined to turn to CD. It is straightforward to 
extend the condition shown in Eq. ^ to the case of multiple CD players. For example, we can similarly 
derive the condition under which player j turns from GC to CD when player i j) is CD and all 
the other N ~ 2 players are GC. For example, on the left-hand side of Eq. we just need to replace 
Ei with Ei + Ej. I confirmed for all the following numerical results that once a player turns from GC 
to CD, some others are also elicited to turn from GC to CD according to the Nash criterion and that 
such a transition from GC to CD cascades until all players are CD. In loose terms, this phenomenon is 
reminiscent of models of cascading failure of overloaded networks, which mimic, for example, blackouts 
on power grids [55] , 

The relationship between wth and c/6 is shown in Fig. [2Ja) for the five networks with A'^ = 20 and 
mean degree fc = 4. The parameter values for the networks are explained in the caption of Fig. [21 A 
small c/b value results in a small Wth value, indicating that cooperation is facilitated. This is generally 
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the case for various mechanisms for cooperation pil37|. 

For reference, the results for direct reciprocity (wth = c/h) and upstream reciprocity on the directed 
triangle (Fig.[TJ w^\^ = \fcj\i) are also shown in Fig.[2l^a) by thin black lines. Except for small cjh values, 
the five networks with iV = 20 possess higher wth values as compared to these reference cases. 

The two networks generated from the WS model yield smaller values of wth than those obtained fr'om 
the RRG, indicating that the WS model allows more cooperation than the RRG. Because the degree 
distributions of these networks are almost the same and the average distances of the RRG and the WS 
model with p = 0.1 do not differ by much [32], I ascribe this difference to clustering. An abundance of 
triangles and short cycles in networks (i.e., the WS model) enhances cooperation. However, the difference 
in Wth is not very large. In quantitative terms, clustering docs little to promote cooperation. 

The same conclusion is supported for heterogeneous networks (the BA and KE models). Values of 
Wth for the KE model, which yields a high level of clustering are smaller than those for the BA model, 
which yields a low level of clustering. However, the wth values for the KE model are considerably larger 
than those for the RRG and the WS model, and the differences between the results for the BA and KE 
models are small. 

To summarize, clustering promotes cooperation but only to a small extent. To further substantiate 
this finding, I looked at different cases. Figure [2Jb) compares wth and cjh values for the networks with 
N — 200 and fc = 6. Figure [3Ia) shows the dependence of u>th on N when cjh = 1/3. These cases also 
suggest that clustering hardly promotes cooperation. 

Scale-free versus homogeneous networks 

Figure [2] indicates that scale-free networks (i.e., the BA and KE models) allow less cooperation than 
networks with a homogeneous degree distribution (i.e., the RRG and WS model). This is in contrast 
with the results for the evolutionary two-person social dilemma games piH27j and those for the evolu- 
tionary upstream reciprocity game [18] on heterogeneous networks in which scale-free networks promote 
cooperation. The difference stems from the fact that players in evolutionary games mimic successful 
neighbors, whereas in my Nash equilibrium model, players judge whether GC or CD is more profitable 
when the other players do not change the strategies (sec Discussion for a more detailed explanation). 

To probe the reason why cooperation is reduced on scale-free networks, I examine the dependence of 
the player-wise threshold value, i.e., Wf^(i) for player i, on node degree fc^. I generate a single network 
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from each of the RRG, the BA model, and the KE model with N = 200 using the same parameter values 
as those used in Fig. ^h). For c/b = 1/3, the relationship between Wth(*) and is shown in Fig. |3]for 
all nodes in the three networks. Wthii) decreases with fc^ in the BA and KE models. In the RRG, fcj is 
equal to 6 for all the nodes, and the value of i«th(*) is approximately the same for all the nodes. 

Wth{i) and fc,; are negatively correlated because the amount of donation flow that a putative CD i 
stops is strongly correlated with Vi. At i = 0, it is equal to Vi. At t > 1, it is generally smaller than Vi, 
but player i having a large w,; value tends to receive a large inflow of donation, which player i stops in the 
next time step. For undirected networks, Vi — kif^^^-^kg oc ki holds true. Flayers with small degrees 
are therefore tempted to convert to CD because the impact of the player's behavior (i.e., to donate or not 
to donate) on the entire network is small. Therefore, a small ki leads to a large z«th(i). Even for directed 
networks, u,; and fc°"* are often strongly correlated [38H40] . Because the minimum degree in a scale- free 
network is smaller than that in a homogeneous network if the mean degree of the two networks is equal, 
scale-free networks have larger wth as compared to homogeneous networks. 

Cooperation in large networks 

A comparison of Figs. [5Ja) and [Hb) suggests that a large N makes cooperation unlikely. To examine 
this point further, I set c/b = 1/3, generated 100 networks for each N value and each network type, 
calculated Wth, and obtained the mean and the standard deviation of wth- Because the WS model with 
p = is unique for a given N, the mean and standard deviation are not relevant in this network. 

The mean and standard deviation of wth for the five networks of various sizes arc shown in Fig. [31Ja). 
The results for the BA and KE models heavily overlap. Cooperation is less likely as N increases in all 
models, except for the WS model with p ~ 0. This result is consistent with that for a directed cycle |13| . 

Wth increases with N not entirely owing to the decreased level of clustering in the network. To show 
this, I plot the mean and standard deviation of the clustering coefficient C(g [0,1]), which quantifies 
the abundance of triangles in a network [321, Fig. ^h). The clustering coefficient is defined as C = 
(1/A^) x^^j^ (number of triangles including node i) /[ki{ki~l) /2]. Figure|3Ib) indicates that C decreases 
with N for the RRG and the BA model. Therefore, the effect of N and C on wth may be mixed in these 
two network models. However, C stays almost constant for the WS and KE models. At least for these 
models, an increase in wth is considered to originate primarily from an increase in N, not from changes 
in the level of clustering. 
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In Fig. ISlJa), Wth seems to approach unity as N increases except for the WS model with p = 0. As 
previously stated, the WS model with p = is unrealistic because it has a large average distance between 
pairs of nodes [201 - 1^ [5^ . Therefore, I conclude that cooperation based on upstream reciprocity is not 
likely for homogeneous and heterogeneous networks in general. 

Discussion 

1 generalized the upstream reciprocity model proposed for a directed cycle [13j to general networks and 
reached two primary conclusions. 

First, cooperation based on upstream reciprocity is not likely in general networks regardless of the 
abundance of triangles and heterogeneity in the node degree. Because the networks that 1 examined 
are undirected, some amount of direct reciprocity is relevant; GC neighbors partially retaliate directly 
against a CD. My result that cooperation is unlikely for undirected networks implies that cooperation 
would be even more difficult for directed networks in which direct reciprocity is less available. In directed 
networks, direct reciprocity occurs only on reciprocal links between a pair of players. 

Second, I showed that scale-free network models allow less cooperation (i.e., large Wth) as compared 
to networks with homogeneous degree distributions. This result is opposite of those for two-person social 
dilemma games p4H27] and the upstream reciprocity game [18] . The difference stems from the fact that 
the previous studies assumed evolutionary games and the present study (and the original model by Boyd 
and Richcrson [13]) is based on nonevolutionary analysis. 

I adopted a nonevolutionary setup and examined the condition for the Nash equilibrium because 
the concept of the evolutionary game on heterogeneous networks seems elusive. Evolutionary games on 
heterogeneous networks imply that a player imitates the strategy of a successful neighbor that is likely to 
have a different node degree. However, players with different degrees are involved in essentially different 
games because the number of times that each player plays the game per generation necessarily depends 
on the degree. Therefore, for example, a small-degree player cannot generally expect a large payoff by 
mimicking a successful neighbor with a large degree. In this situation, defining the game and payoff for 
players with various degrees is complicated [26l[28l[29] . Use of the Nash criterion does not incur this type 
of problem. 

The overall conclusions of the present study are negative. To explain the occurrence of upstream 
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reciprocity in real societies, it may be advantageous to combine upstream reciprocity with other non- 
network mechanisms, such as the ones mentioned in the Introduction. 



Methods 

Numerical methods for calculating Eqs. and f ITT]) 

I determined Wth by applying the bisection method to Eq. ^ or ((TT]). To calculate [/ — w{I — Ei)D~^A\ ^ 
for different values of w, it is beneficial to use the expansion of (/ — Ei)D^^A in terms of independent 
modes. This is possible when the adjacency matrix A for the subnetwork composed of the GCs is diago- 
nalizable, as shown below. 

I assume that there are = N — GCs and A^d CDs. In the main text, I focused on the case 
Nd — 1. However, the case > 2 is also relevant because I verified in the main text that the appearance 
of a single CD leads to the further emergence of CDs. Without loss of generality, I assume that players 1, 
2, . . . , Ns are GC and players Ng + l, Ns + 2, . . ., N are CD, and that the network is strongly connected. 
We need to identify all the (generalized) eigenmodes of E^D^^A, where 

E, = Y,E^. (16) 
1=1 

I first partition Eg, D^^, and A into two-by-two blocks, each partition corresponding to the set of GC 
and that of CD. For a candidate of a left eigenvector of EgD^^A, denoted by 



v'^'^E,D-'A={vP vl;' 



.W n-i 





:) 


o \ 


(a. 


Asd^ 


\° 






[^Ads 


^ddy 






.). 







(17) 



where In^ is the identity matrix of size Ng] Dg and are diagonal matrices whose diagonal entries are 
equal to the outdegrees of the GCs and CDs, respectively; Ags is the Ng-hy-Ng matrix corresponding to 
the adjacent matrix within the GCs; and Agd, ^ds, and A^d are similarly defined blocks of the original 
adjacency matrix A. Note that A^g and A^d are absent on the right-hand side of Eq. p7|) and as such 
are not relevant to the following discussion. 
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First of all, v^'^ = {vi^^ v^^) = ej {Ns + 1 < i < N) is a. trivial zero left eigenvector of EsD^^A. 
Here, T denotes transpose, and is the unit column vector in which the ith element is equal to unity 
and all the other elements are equal to zero. 

To obtain the other Ng generalized eigenmodes of EgD^^A, I consider the case in which D^^Ag^ is 
diagonalizable. Otherwise, efficiently calculating [/ — wEsD^^A\ ^ via matrix decomposition is difficult. 
D^^Ass is diagonalizable if the network is undirected. A diagonalizable D~^Ass possesses nondegen- 
erate left eigenvector Vs*'' (1 < « < Ng) with the corresponding eigenvalue A^. It is possible that — Xj 
for i ^ j. 

If Xi 7^ 0, A,; is an eigenvalue of EsD~^A, and the corresponding left eigenvector is given by = 
(vi^^ v'^^), where 

= ^l^^. (18) 
Xi 

If Xi = 0, Eq. ([T7|) implies that u^*) = (vg''' v^*'') is not a left eigenvector of EsD~^A. An example 
network with = 3 that has nontrivial zero eigenvalues is presented in the next section for a pedagogical 
purpose. When A^ =0, I set =0 such that 

v^'^E,D-'A = ( ^ vi^'^D^^A,^). (19) 

Ns zeros 

Because (0 vi'^D-^A,d) can be represented as a linear sum of eJ {Ng + 1 < i < A^), f is a type of 
generalized eigenvector corresponding to A; = 0. 

I denote by it*^*^ (1 < « < A"s) the nontrivial generalized right eigenmodes of EsD~^A corresponding 
to v'-'^K To obtain m'*', I denote by its*'' (1 < i < A^s) the normalized right eigenvectors of D^^Ags with 
eigenvalue A,;. Then, 



/ u.^'A }size A^s 

{l<i< AT,) (20) 

}A^d zeros 



V / 

are right eigenvectors of EsD^^A that respect the orthogonality ii, where S is the Kronecker 

delta. 

For completeness, I obtain the expression of the other Ad right eigenvectors of EsD~^A corresponding 
to the trivial zero eigenvalue as follows. I align d^*) and m'*^ (1 < « < A"s) such that nonzero eigenvectors 
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correspond to 1 < i < A'g — A'o and generalized zero eigenmodes correspond to A's — A^'o + 1 < « < A's 
Then, the orthogonahty condition v'^^^u'^^^ = 6ij reads 



Ai 



(N,-No + l) 



V 



(21) 



for an Ns-hy-N^ matrix M. Equation (I^Tt yields 



Ai 



\ AiVa-No 



(22) 



Finally, the decomposition oi EgD is given by 
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Alt); 



(1) 



, {N,-N„) „W-JVo)r)-l /I 



sd 



J=i I 



0) u'^'n-^A 



v'-^^D-^A.d 



(23) 



Combining Eq. (|23p and the orthogonality condition v^^-'ms''-' = I obtain 



V 



>2). 



(24) 



Using Eqs. (fT6|) . (|23|) . and (f24| . we can express the quantities appearing on the left-hand sides of 
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Eqs. dH) and ^ as 



[l-w{I-E,)D-'A]- =/+ E I 



w 



A. 



wX 



J V g — i V Q 



Af= /it 

E 



V 

0)' 



A, 



wA," 



v 







JV3 /m 



or 



-El I ( v^^D^^A,, 



(25) 



(26) 



If Ass is symmetric, Ds ^^^A^sDs is also symmetric and therefore diagonalizable by a unitary ma- 

— 1/2 — 1/2 ^ f" '"i 

trix. Denote the eigenvalue and the right eigenvector of Ds AssDs by Xi and u^^> , respectively. 

Note that A^ and ■u^'-' are both real and can be computed relatively easily. Then, we can obtain the re- 
lationships Xi = Xi, tti*'' = Ds ^^^■u''', and vi'' = u'^'''^ dI^^ . We can also obtain = ■u'^-'^Ds '^^^Asr/Xi 
when Xi ^ 0. 



Example network yielding nontrivial zero eigenmodes 

Consider the undirected network having TV = 3 nodes as shown in Fig. [5] For this network I obtain 





^0 


l\ 


D-^A = 





1 




\2 2 





(27) 



By turning player 3 from GC to CD, I obtain 



(/ - E3)D-^A = {El + E2)D-^A 



1 
yO Oy 



(28) 
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All of the eigenvalues of matrix (|28p are equal to zero, one trivial and two nontrivial. The one trivial zero 
eigenvalue originates from removing player 3 from the network of GCs. The trivial zero left eigenvector 
is given by ^3 = ej[ = (0 1). I select the two generalized zero left eigenmodes to be Vi ~ ej {i — 1,2). 
The choice of vi and V2 is not unique. The right eigenmodes are given by Ui = e,(l < i < 3). 

Equation ([T9l). for example, then reads Vi{Ei + £2)0^^ A = V2{Ei + £2)0^^ A = V3 and V3{Ei + 
E2)D-^A = 0. 
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Figure 1. Directed cycle with = 3 nodes. 
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Figure 2. Relationship between threshold discount factor (wth) and cost-to-benefit ratio 

(c/6). I use the five types of networks and set (a) TV = 20, k = 4, and (b) N = 200, fc = 6. The results 
for direct reciprocity (i.e., wth ~ c/b) and upstream reciprocity on the directed triangle (i.e., 
Wth = \/ c/b) are also shown by thin black lines for comparison. In (a), I set the rewiring probability for 
the WS model to p = and p = 0.1. For the BA model, there are initially toq = 2 nodes (i.e., dyad), 
and the number of links that each added node has is set to m = 2. For my variant of the KE model, the 
initial number of nodes and the number of links that each added node has are set to to = 2, and an 
active node i is deactivated with probability proportional to (fc^ + a)~^, where a = 2. After constructing 
the network based on the original KE model [34], I rewire fraction p = 0.1 of randomly selected links to 
make the average distance small. In (b), I set p = and p = 0.05 for the WS model, toq = to = 3 for 
the BA model, and to = a = 3 and p = 0.05 for the KE model. 
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Figure 3. Effects of network size {N). (a) Dependence of the threshold discount factor (wth) on N. 
(b) Dependence of the clustering coefficient (C) on N. I use the five types of networks and set 
c/b = 1/3. The parameter values for the networks are the same as those used in Fig.[2][b). In (a), the 
results for the BA and KE models heavily overlap. 
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Figure 4. Relationship between threshold discount factor (u'th(j)) and node degree (fc^). 

use the RRG, the BA model, and the KE model with N ^ 200 and fc = 6, and set c/b = 1/3. The 
parameter values for the networks are the same as those used in Fig. [2jb). 
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Figure 5. A network yielding nontrivial zero eigenval 



